<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>General Techniques</title>
<link rel="stylesheet" href="../../boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.79.1">
<link rel="home" href="../index.html" title="Boost.Python Tutorial">
<link rel="up" href="../index.html" title="Boost.Python Tutorial">
<link rel="prev" href="exception.html" title="Exception Translation">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr><td valign="top"><img alt="" width="" height="" src="../../images/boost.png"></td></tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="exception.html"><img src="../../images/prev.png" alt="Prev"></a><a accesskey="u" href="../index.html"><img src="../../images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../images/home.png" alt="Home"></a>
</div>
<div class="section">
<div class="titlepage"><div><div><h2 class="title" style="clear: both">
<a name="tutorial.techniques"></a><a class="link" href="techniques.html" title="General Techniques">General Techniques</a>
</h2></div></div></div>
<div class="toc"><dl class="toc">
<dt><span class="section"><a href="techniques.html#tutorial.techniques.creating_packages">Creating Packages</a></span></dt>
<dt><span class="section"><a href="techniques.html#tutorial.techniques.extending_wrapped_objects_in_pyt">Extending
      Wrapped Objects in Python</a></span></dt>
<dt><span class="section"><a href="techniques.html#tutorial.techniques.reducing_compiling_time">Reducing
      Compiling Time</a></span></dt>
</dl></div>
<p>
      Here are presented some useful techniques that you can use while wrapping code
      with Boost.Python.
    </p>
<div class="section">
<div class="titlepage"><div><div><h3 class="title">
<a name="tutorial.techniques.creating_packages"></a><a class="link" href="techniques.html#tutorial.techniques.creating_packages" title="Creating Packages">Creating Packages</a>
</h3></div></div></div>
<p>
        A Python package is a collection of modules that provide to the user a certain
        functionality. If you're not familiar on how to create packages, a good introduction
        to them is provided in the <a href="http://www.python.org/doc/current/tut/node8.html" target="_top">Python
        Tutorial</a>.
      </p>
<p>
        But we are wrapping C++ code, using Boost.Python. How can we provide a nice
        package interface to our users? To better explain some concepts, let's work
        with an example.
      </p>
<p>
        We have a C++ library that works with sounds: reading and writing various
        formats, applying filters to the sound data, etc. It is named (conveniently)
        <code class="literal">sounds</code>. Our library already has a neat C++ namespace hierarchy,
        like so:
      </p>
<pre class="programlisting"><span class="identifier">sounds</span><span class="special">::</span><span class="identifier">core</span>
<span class="identifier">sounds</span><span class="special">::</span><span class="identifier">io</span>
<span class="identifier">sounds</span><span class="special">::</span><span class="identifier">filters</span>
</pre>
<p>
        We would like to present this same hierarchy to the Python user, allowing
        him to write code like this:
      </p>
<pre class="programlisting"><span class="keyword">import</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span>
<span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span><span class="special">.</span><span class="identifier">echo</span><span class="special">(...)</span> <span class="comment"># echo is a C++ function</span>
</pre>
<p>
        The first step is to write the wrapping code. We have to export each module
        separately with Boost.Python, like this:
      </p>
<pre class="programlisting"><span class="special">/*</span> <span class="identifier">file</span> <span class="identifier">core</span><span class="special">.</span><span class="identifier">cpp</span> <span class="special">*/</span>
<span class="identifier">BOOST_PYTHON_MODULE</span><span class="special">(</span><span class="identifier">core</span><span class="special">)</span>
<span class="special">{</span>
    <span class="special">/*</span> <span class="identifier">export</span> <span class="identifier">everything</span> <span class="keyword">in</span> <span class="identifier">the</span> <span class="identifier">sounds</span><span class="special">::</span><span class="identifier">core</span> <span class="identifier">namespace</span> <span class="special">*/</span>
    <span class="special">...</span>
<span class="special">}</span>

<span class="special">/*</span> <span class="identifier">file</span> <span class="identifier">io</span><span class="special">.</span><span class="identifier">cpp</span> <span class="special">*/</span>
<span class="identifier">BOOST_PYTHON_MODULE</span><span class="special">(</span><span class="identifier">io</span><span class="special">)</span>
<span class="special">{</span>
    <span class="special">/*</span> <span class="identifier">export</span> <span class="identifier">everything</span> <span class="keyword">in</span> <span class="identifier">the</span> <span class="identifier">sounds</span><span class="special">::</span><span class="identifier">io</span> <span class="identifier">namespace</span> <span class="special">*/</span>
    <span class="special">...</span>
<span class="special">}</span>

<span class="special">/*</span> <span class="identifier">file</span> <span class="identifier">filters</span><span class="special">.</span><span class="identifier">cpp</span> <span class="special">*/</span>
<span class="identifier">BOOST_PYTHON_MODULE</span><span class="special">(</span><span class="identifier">filters</span><span class="special">)</span>
<span class="special">{</span>
    <span class="special">/*</span> <span class="identifier">export</span> <span class="identifier">everything</span> <span class="keyword">in</span> <span class="identifier">the</span> <span class="identifier">sounds</span><span class="special">::</span><span class="identifier">filters</span> <span class="identifier">namespace</span> <span class="special">*/</span>
    <span class="special">...</span>
<span class="special">}</span>
</pre>
<p>
        Compiling these files will generate the following Python extensions: <code class="literal">core.pyd</code>,
        <code class="literal">io.pyd</code> and <code class="literal">filters.pyd</code>.
      </p>
<div class="note"><table border="0" summary="Note">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../images/note.png"></td>
<th align="left">Note</th>
</tr>
<tr><td align="left" valign="top"><p>
          The extension <code class="literal">.pyd</code> is used for python extension modules,
          which are just shared libraries. Using the default for your system, like
          <code class="literal">.so</code> for Unix and <code class="literal">.dll</code> for Windows,
          works just as well.
        </p></td></tr>
</table></div>
<p>
        Now, we create this directory structure for our Python package:
      </p>
<pre class="programlisting">sounds/
    __init__.py
    core.pyd
    filters.pyd
    io.pyd
</pre>
<p>
        The file <code class="literal">__init__.py</code> is what tells Python that the directory
        <code class="literal">sounds/</code> is actually a Python package. It can be a empty
        file, but can also perform some magic, that will be shown later.
      </p>
<p>
        Now our package is ready. All the user has to do is put <code class="literal">sounds</code>
        into his <a href="http://www.python.org/doc/current/tut/node8.html#SECTION008110000000000000000" target="_top">PYTHONPATH</a>
        and fire up the interpreter:
      </p>
<pre class="programlisting"><span class="special">&gt;&gt;&gt;</span> <span class="keyword">import</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">io</span>
<span class="special">&gt;&gt;&gt;</span> <span class="keyword">import</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">sound</span> <span class="special">=</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">io</span><span class="special">.</span><span class="identifier">open</span><span class="special">(</span><span class="string">'file.mp3'</span><span class="special">)</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">new_sound</span> <span class="special">=</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span><span class="special">.</span><span class="identifier">echo</span><span class="special">(</span><span class="identifier">sound</span><span class="special">,</span> <span class="number">1.0</span><span class="special">)</span>
</pre>
<p>
        Nice heh?
      </p>
<p>
        This is the simplest way to create hierarchies of packages, but it is not
        very flexible. What if we want to add a <span class="emphasis"><em>pure</em></span> Python
        function to the filters package, for instance, one that applies 3 filters
        in a sound object at once? Sure, you can do this in C++ and export it, but
        why not do so in Python? You don't have to recompile the extension modules,
        plus it will be easier to write it.
      </p>
<p>
        If we want this flexibility, we will have to complicate our package hierarchy
        a little. First, we will have to change the name of the extension modules:
      </p>
<pre class="programlisting"><span class="comment">/* file core.cpp */</span>
<span class="identifier">BOOST_PYTHON_MODULE</span><span class="special">(</span><span class="identifier">_core</span><span class="special">)</span>
<span class="special">{</span>
    <span class="special">...</span>
    <span class="comment">/* export everything in the sounds::core namespace */</span>
<span class="special">}</span>
</pre>
<p>
        Note that we added an underscore to the module name. The filename will have
        to be changed to <code class="literal">_core.pyd</code> as well, and we do the same
        to the other extension modules. Now, we change our package hierarchy like
        so:
      </p>
<pre class="programlisting">sounds/
    __init__.py
    core/
        __init__.py
        _core.pyd
    filters/
        __init__.py
        _filters.pyd
    io/
        __init__.py
        _io.pyd
</pre>
<p>
        Note that we created a directory for each extension module, and added a __init__.py
        to each one. But if we leave it that way, the user will have to access the
        functions in the core module with this syntax:
      </p>
<pre class="programlisting"><span class="special">&gt;&gt;&gt;</span> <span class="keyword">import</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">core</span><span class="special">.</span><span class="identifier">_core</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">core</span><span class="special">.</span><span class="identifier">_core</span><span class="special">.</span><span class="identifier">foo</span><span class="special">(...)</span>
</pre>
<p>
        which is not what we want. But here enters the <code class="literal">__init__.py</code>
        magic: everything that is brought to the <code class="literal">__init__.py</code> namespace
        can be accessed directly by the user. So, all we have to do is bring the
        entire namespace from <code class="literal">_core.pyd</code> to <code class="literal">core/__init__.py</code>.
        So add this line of code to <code class="literal">sounds/core/__init__.py</code>:
      </p>
<pre class="programlisting"><span class="keyword">from</span> <span class="identifier">_core</span> <span class="keyword">import</span> <span class="special">*</span>
</pre>
<p>
        We do the same for the other packages. Now the user accesses the functions
        and classes in the extension modules like before:
      </p>
<pre class="programlisting"><span class="special">&gt;&gt;&gt;</span> <span class="keyword">import</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span><span class="special">.</span><span class="identifier">echo</span><span class="special">(...)</span>
</pre>
<p>
        with the additional benefit that we can easily add pure Python functions
        to any module, in a way that the user can't tell the difference between a
        C++ function and a Python function. Let's add a <span class="emphasis"><em>pure</em></span>
        Python function, <code class="literal">echo_noise</code>, to the <code class="literal">filters</code>
        package. This function applies both the <code class="literal">echo</code> and <code class="literal">noise</code>
        filters in sequence in the given <code class="literal">sound</code> object. We create
        a file named <code class="literal">sounds/filters/echo_noise.py</code> and code our
        function:
      </p>
<pre class="programlisting"><span class="keyword">import</span> <span class="identifier">_filters</span>
<span class="keyword">def</span> <span class="identifier">echo_noise</span><span class="special">(</span><span class="identifier">sound</span><span class="special">):</span>
    <span class="identifier">s</span> <span class="special">=</span> <span class="identifier">_filters</span><span class="special">.</span><span class="identifier">echo</span><span class="special">(</span><span class="identifier">sound</span><span class="special">)</span>
    <span class="identifier">s</span> <span class="special">=</span> <span class="identifier">_filters</span><span class="special">.</span><span class="identifier">noise</span><span class="special">(</span><span class="identifier">sound</span><span class="special">)</span>
    <span class="keyword">return</span> <span class="identifier">s</span>
</pre>
<p>
        Next, we add this line to <code class="literal">sounds/filters/__init__.py</code>:
      </p>
<pre class="programlisting"><span class="keyword">from</span> <span class="identifier">echo_noise</span> <span class="keyword">import</span> <span class="identifier">echo_noise</span>
</pre>
<p>
        And that's it. The user now accesses this function like any other function
        from the <code class="literal">filters</code> package:
      </p>
<pre class="programlisting"><span class="special">&gt;&gt;&gt;</span> <span class="keyword">import</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">sounds</span><span class="special">.</span><span class="identifier">filters</span><span class="special">.</span><span class="identifier">echo_noise</span><span class="special">(...)</span>
</pre>
</div>
<div class="section">
<div class="titlepage"><div><div><h3 class="title">
<a name="tutorial.techniques.extending_wrapped_objects_in_pyt"></a><a class="link" href="techniques.html#tutorial.techniques.extending_wrapped_objects_in_pyt" title="Extending Wrapped Objects in Python">Extending
      Wrapped Objects in Python</a>
</h3></div></div></div>
<p>
        Thanks to Python's flexibility, you can easily add new methods to a class,
        even after it was already created:
      </p>
<pre class="programlisting"><span class="special">&gt;&gt;&gt;</span> <span class="keyword">class</span> <span class="identifier">C</span><span class="special">(</span><span class="identifier">object</span><span class="special">):</span> <span class="keyword">pass</span>
<span class="special">&gt;&gt;&gt;</span>
<span class="special">&gt;&gt;&gt;</span> <span class="comment"># a regular function</span>
<span class="special">&gt;&gt;&gt;</span> <span class="keyword">def</span> <span class="identifier">C_str</span><span class="special">(</span><span class="identifier">self</span><span class="special">):</span> <span class="keyword">return</span> <span class="string">'A C instance!'</span>
<span class="special">&gt;&gt;&gt;</span>
<span class="special">&gt;&gt;&gt;</span> <span class="comment"># now we turn it in a member function</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">C</span><span class="special">.</span><span class="identifier">__str__</span> <span class="special">=</span> <span class="identifier">C_str</span>
<span class="special">&gt;&gt;&gt;</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">c</span> <span class="special">=</span> <span class="identifier">C</span><span class="special">()</span>
<span class="special">&gt;&gt;&gt;</span> <span class="keyword">print</span> <span class="identifier">c</span>
<span class="identifier">A</span> <span class="identifier">C</span> <span class="identifier">instance</span><span class="special">!</span>
<span class="special">&gt;&gt;&gt;</span> <span class="identifier">C_str</span><span class="special">(</span><span class="identifier">c</span><span class="special">)</span>
<span class="identifier">A</span> <span class="identifier">C</span> <span class="identifier">instance</span><span class="special">!</span>
</pre>
<p>
        Yes, Python rox. <span class="inlinemediaobject"><img src="../../images/smiley.png"></span>
      </p>
<p>
        We can do the same with classes that were wrapped with Boost.Python. Suppose
        we have a class <code class="literal">point</code> in C++:
      </p>
<pre class="programlisting"><span class="keyword">class</span> <span class="identifier">point</span> <span class="special">{...};</span>

<span class="identifier">BOOST_PYTHON_MODULE</span><span class="special">(</span><span class="identifier">_geom</span><span class="special">)</span>
<span class="special">{</span>
    <span class="identifier">class_</span><span class="special">&lt;</span><span class="identifier">point</span><span class="special">&gt;(</span><span class="string">"point"</span><span class="special">)...;</span>
<span class="special">}</span>
</pre>
<p>
        If we are using the technique from the previous session, <a class="link" href="techniques.html#tutorial.techniques.creating_packages" title="Creating Packages">Creating
        Packages</a>, we can code directly into <code class="literal">geom/__init__.py</code>:
      </p>
<pre class="programlisting"><span class="keyword">from</span> <span class="identifier">_geom</span> <span class="keyword">import</span> <span class="special">*</span>

<span class="comment"># a regular function</span>
<span class="keyword">def</span> <span class="identifier">point_str</span><span class="special">(</span><span class="identifier">self</span><span class="special">):</span>
    <span class="keyword">return</span> <span class="identifier">str</span><span class="special">((</span><span class="identifier">self</span><span class="special">.</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">self</span><span class="special">.</span><span class="identifier">y</span><span class="special">))</span>

<span class="comment"># now we turn it into a member function</span>
<span class="identifier">point</span><span class="special">.</span><span class="identifier">__str__</span> <span class="special">=</span> <span class="identifier">point_str</span>
</pre>
<p>
        <span class="bold"><strong>All</strong></span> point instances created from C++ will
        also have this member function! This technique has several advantages:
      </p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem">
            Cut down compile times to zero for these additional functions
          </li>
<li class="listitem">
            Reduce the memory footprint to virtually zero
          </li>
<li class="listitem">
            Minimize the need to recompile
          </li>
<li class="listitem">
            Rapid prototyping (you can move the code to C++ if required without changing
            the interface)
          </li>
</ul></div>
<p>
        Another useful idea is to replace constructors with factory functions:
      </p>
<pre class="programlisting"><span class="identifier">_point</span> <span class="special">=</span> <span class="identifier">point</span>

<span class="keyword">def</span> <span class="identifier">point</span><span class="special">(</span><span class="identifier">x</span><span class="special">=</span><span class="number">0</span><span class="special">,</span> <span class="identifier">y</span><span class="special">=</span><span class="number">0</span><span class="special">):</span>
    <span class="keyword">return</span> <span class="identifier">_point</span><span class="special">(</span><span class="identifier">x</span><span class="special">,</span> <span class="identifier">y</span><span class="special">)</span>
</pre>
<p>
        In this simple case there is not much gained, but for constructurs with many
        overloads and/or arguments this is often a great simplification, again with
        virtually zero memory footprint and zero compile-time overhead for the keyword
        support.
      </p>
</div>
<div class="section">
<div class="titlepage"><div><div><h3 class="title">
<a name="tutorial.techniques.reducing_compiling_time"></a><a class="link" href="techniques.html#tutorial.techniques.reducing_compiling_time" title="Reducing Compiling Time">Reducing
      Compiling Time</a>
</h3></div></div></div>
<p>
        If you have ever exported a lot of classes, you know that it takes quite
        a good time to compile the Boost.Python wrappers. Plus the memory consumption
        can easily become too high. If this is causing you problems, you can split
        the class_ definitions in multiple files:
      </p>
<pre class="programlisting"><span class="comment">/* file point.cpp */</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">point</span><span class="special">.</span><span class="identifier">h</span><span class="special">&gt;</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">python</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>

<span class="keyword">void</span> <span class="identifier">export_point</span><span class="special">()</span>
<span class="special">{</span>
    <span class="identifier">class_</span><span class="special">&lt;</span><span class="identifier">point</span><span class="special">&gt;(</span><span class="string">"point"</span><span class="special">)...;</span>
<span class="special">}</span>

<span class="comment">/* file triangle.cpp */</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">triangle</span><span class="special">.</span><span class="identifier">h</span><span class="special">&gt;</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">python</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>

<span class="keyword">void</span> <span class="identifier">export_triangle</span><span class="special">()</span>
<span class="special">{</span>
    <span class="identifier">class_</span><span class="special">&lt;</span><span class="identifier">triangle</span><span class="special">&gt;(</span><span class="string">"triangle"</span><span class="special">)...;</span>
<span class="special">}</span>
</pre>
<p>
        Now you create a file <code class="literal">main.cpp</code>, which contains the <code class="literal">BOOST_PYTHON_MODULE</code>
        macro, and call the various export functions inside it.
      </p>
<pre class="programlisting"><span class="keyword">void</span> <span class="identifier">export_point</span><span class="special">();</span>
<span class="keyword">void</span> <span class="identifier">export_triangle</span><span class="special">();</span>

<span class="identifier">BOOST_PYTHON_MODULE</span><span class="special">(</span><span class="identifier">_geom</span><span class="special">)</span>
<span class="special">{</span>
    <span class="identifier">export_point</span><span class="special">();</span>
    <span class="identifier">export_triangle</span><span class="special">();</span>
<span class="special">}</span>
</pre>
<p>
        Compiling and linking together all this files produces the same result as
        the usual approach:
      </p>
<pre class="programlisting"><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">python</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">point</span><span class="special">.</span><span class="identifier">h</span><span class="special">&gt;</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">triangle</span><span class="special">.</span><span class="identifier">h</span><span class="special">&gt;</span>

<span class="identifier">BOOST_PYTHON_MODULE</span><span class="special">(</span><span class="identifier">_geom</span><span class="special">)</span>
<span class="special">{</span>
    <span class="identifier">class_</span><span class="special">&lt;</span><span class="identifier">point</span><span class="special">&gt;(</span><span class="string">"point"</span><span class="special">)...;</span>
    <span class="identifier">class_</span><span class="special">&lt;</span><span class="identifier">triangle</span><span class="special">&gt;(</span><span class="string">"triangle"</span><span class="special">)...;</span>
<span class="special">}</span>
</pre>
<p>
        but the memory is kept under control.
      </p>
<p>
        This method is recommended too if you are developing the C++ library and
        exporting it to Python at the same time: changes in a class will only demand
        the compilation of a single cpp, instead of the entire wrapper code.
      </p>
<div class="note"><table border="0" summary="Note">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../images/note.png"></td>
<th align="left">Note</th>
</tr>
<tr><td align="left" valign="top"><p>
          This method is useful too if you are getting the error message <span class="emphasis"><em>"fatal
          error C1204:Compiler limit:internal structure overflow"</em></span>
          when compiling a large source file, as explained in the <a href="../../faq/fatal_error_c1204_compiler_limit.html" target="_top">FAQ</a>.
        </p></td></tr>
</table></div>
</div>
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright © 2002-2005 Joel
      de Guzman, David Abrahams<p>
        Distributed under the Boost Software License, Version 1.0. (See accompanying
        file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>
      </p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="exception.html"><img src="../../images/prev.png" alt="Prev"></a><a accesskey="u" href="../index.html"><img src="../../images/up.png" alt="Up"></a><a accesskey="h" href="../index.html"><img src="../../images/home.png" alt="Home"></a>
</div>
</body>
</html>
